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Introduction 

In recent years, maintenance organizations have been interested in increasing their 
ability to leam from the investigation and analysis of incident and accident events. 
However, they have found it difficult to move from the compliance-based traditional 
mode of error management, to more voluntary (and less punishment oriented) 
participatory investigation programs. David Marx has coined the useful terms “rule- 
based” and “risk-based” to refer to these two modes of error management (2001). 

The aviation maintenance community is at a crossroads with respect to 
implementing the Aviation Safety Action Program (ASAP). While there is considerable 
interest, several key issues have emerged that cast doubt on how to assure a successful 
implementation, including buy-in from all levels of the company and training for key 
participants. There are two objectives for the present report. The first is to provide an 
examination of limits (or more properly, examples) of the degree of acceptability of more 
problematic events for risk-based decisions within the current ASAP guidelines. The 
second objective is to apply these limits of “community standards” to a set of further 
refined ASAP training scenarios. 

In part, a successful maintenance ASAP implementation depends on a robust 
Memorandum of Understanding (MOU) and Event Review Committee (ERC) process, 
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both of which require clear criteria for deciding whether individual cases are accepted or 
excluded. Potential inconsistencies brought about by differences in rule-based vs. risk- 
based philosophies need to be resolved so all key participants share a formalized, 
common, understanding. Training materials consistent with community standards of 
acceptable ASAP events should help form more readily enunciated criteria for improving 
present practice, as well as improving diffusion of further programs 

Exploring reject/accept criteria. 

Embedded within the current ASAP guidelines, as described in AC 120-66B 
(FAA, 2002), are a number of criteria for excluding a submission to the program. Among 
these are rule-based criteria such as an untimely report (one submitted beyond a pre-set 
time limit unless it is a sole source report), a violation committed while abusing 
controlled substances or alcohol, or a violation committed while engaged in criminal 
activity. There are also “risk-based” criteria to be found in AC 120-66B, which can be 
subsumed within the construct, “intentional (or ‘not inadvertent’) disregard for safety.” 
There are no fixed definitions or unequivocal boundaries for intentional/unintentional 
disregard for safety. Deciding that the reporter’s action was unintentional with regard to 
safety is a matter of consensus for an ERC operating under an approved ASAP MOU. 

The ERC, when the members decide to accept a report under ASAP protection, in fact 
acknowledges the risk or uncertainty that the reporter did not purposefully disregard 
safety. Both rule-based and risk-based allusions found within AC 120-66B will make 
their way into the MOU of any ASAP program, where they will henceforth be applied by 
the ERC that created them. 
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The impact of sole source reports in Maintenance ASAP. According to AC 120-66B if 
an excluded sole source report (i.e. all evidence of the event is based or predicted upon 
the report) is referred to the FAA, the agency would have no basis for enforcement action 
because it would have no independent evidence. Similarly a company cannot use an 
excluded sole source report as a basis for company discipline. On the other hand, those 
reports submitted to a Maintenance ASAP ERC that are excluded, and are not sole 
source, may be subject to legal enforcement action. When this is the case the FAA 
enforcement or company disciplinary action can only be taken based on independent 
evidence of the event (i.e., evidence not based or predicated on the report itself). Thus 
punishment is unlikely if a mechanic unsuccessfully reports what he/she believes is an 
unintentional error. 

There is a complication however. In aircraft maintenance (though unlikely in 
flight operations) the possibility exists that there will be independent evidence of the 
reported event, which is at least available to the company, though not necessarily to the 
FAA. Thus it is possible that a mechanic reporting an error that is excluded from ASAP 
might be subject to punishment if and when additional evidence for that error becomes 
available. 

Risk-based approach to protected disclosure in Maintenance ASAP. Any ERC 
exploring risk-based cases must rely on their operational definition of what is acceptable 
and what should be excluded. The added complication of possible punishment for those 
maintenance personnel whose report may be excluded — even though presumed sole- 
source — must be considered. Risk-based decisions must therefore take the special nature 
of maintenance errors into account. 
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In the case of excluded cases the ERC may find it difficult to define or apply a 
definition of ‘intentional disregard for safety,’ but they “will know it when they see it” 
(acknowledging the words of Supreme Court Justice Potter Stewart). 

In 1964, Justice Potter Stewart tried to explain "hard-core" pornography, or what 
is obscene, by saying, "I shall not today attempt further to define the kinds of material I 
understand to be embraced . . . [b]ut I know it when I see it . . . " (Jacobellis v. Ohio, 378 
U.S. 184, 197; 1964). This quote, and the intent behind it, is well known as summarizing 
the irony and difficulty in trying to define the concept of “obscenity.” For Maintenance 
ERCs considering a risk-based approach to aviation safety investigations, “intentional 
disregard for safety” is as difficult a concept as obscenity to define. 

Community Standards. Maintenance ASAP ERC members defining “intentional 
disregard for safety,” on a case by case basis while exercising risk-based philosophy of 
error management, are applying their " community standards " — just as in the area of 
pornography for jurors, "community standards" are the standards of the area "from which 
[the juror] comes for making the required " definition of obscenity (Handing v. United 
States, 418 U.S. 87, 105; 1974). 

The Survey. Nineteen maintenance cases were derived from public files, such as 
the NASA Aviation Safety Reporting System (ASRS), which illustrate deviation from 14 
CFR (violations) or from safe practice (at-risk behaviors). If there is no apparent 
violation of 14 CFR, unsafe practice can be assumed by virtue of the self-report being 
voluntarily submitted to ASRS by the reporter. Both potential violations and risk-based 
cases selected and used included some aspect of direct “social pressure,” or the indirect 
effect of social norms within the existing social context. All 19 scenarios were selected 
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and developed as potential comparison-points to the exclusionary case illustration in AC 
120-66B (FAA, 2002; p 13). That case - - describing a mechanic's use of a substitute 
engine valve lubricant at the direction of his/her immediate supervisor, with full 
knowledge of both parties that the lubricant was not authorized in the maintenance 
manual - - illustrates a non-sole-source report that the ERC concluded involved an 
intentional disregard for safety warranting referral of the event to the FAA for possible 
enforcement action against both the reporting employee and his/her supervisor. If 
community standards in maintenance match the standards of those who created the case 
illustration in AC 120-66B, then this study should reveal that, by showing exclusion for 
similar applications of social pressure for unsafe maintenance practices known by one or 
both parties. 

Survey data and the analysis. Data were collected during late 2003 and early 2004 
from 19 individuals who comprised about two-thirds of the nationwide population of 
Maintenance ASAP practitioners and those currently developing a Maintenance ASAP 
program. Those data were examined to better understand the community standard for 
intentional disregard for safety. The results show that intentional disregard for safety was 
not seen to be a factor, by most of the survey respondents in any of the cases. In other 
words a community standard was revealed that seems to directly counter the exclusionary 
illustration in AC 120-66B. 

Selection and development of the training scenarios. The data analysis showed 
that seven of the 19 scenarios would be most appropriate candidates for inclusion in 
ASAP training for ERC members. That selection was based on the content of the 
scenarios (acquiescence to supervisory pressure); as well as on their ability to elicit 


5 



variability in response, while at the same time retaining the overall community standards 
for protected disclosure and organizational learning within this social context. 


The Research 

Research Question: “How acceptable to ASAP are those maintenance human errors, 
caused by pressure in the system (e.g., from supervisor, from coworkers, from 
management, or from standard norms of “how we do things around here”). 

Our study permitted testing the following specific parts of that question: 

1 . Concept : Do the roles taken by the ERC members influence acceptability of 
mistakes and violations Aviation Maintenance Technicians (AMTs) report to an 
ASAP program? 

a. Operational Definition : Are there intergroup differences between 
management, union and FAA representatives in their responses to the 
scenarios? 

2. Concept: What is the acceptability of mistakes and violations Aviation 
Maintenance Technicians (AMTs) report to an ASAP program when they report 
that act was influenced by expectations from supervisor or management, from 
Lead Mechanics, or from coworkers for nonnative or typical behavior? 

a. Operational Definition: What is (are) the dominant response(s) to all of 
the scenarios? 

b. Operational Definition: Are there differences in response to the individual 
scenarios? 

i. How much unanimity among respondents (Rs) is there for each 
scenario? 

ii. What characterizes those scenarios for which unanimity is higher? 

c. Operational Definition: Are there patterns of individual response to 
scenarios illustrating certain kinds of social expectation or pressure? 
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Method 


Instrument : A 19 item multiple choice survey was constructed. Each item provided a 
variation of a hypothetical AMT’s submission to a maintenance ASAP program. These 
hypothetical submissions, or scenarios, were developed from cases found in the public 
files of NASA’s Aviation Safety Reporting System (ASRS), or in the public archives of 
the US National Transportation Safety Board (NTSB) and are intended to represent 
realistic scenarios. All these scenarios represent some version of a mistake or human 
error within a direct social context. These social situations include communication 
(usually in form of instruction) to the AMT from his/her management supervisor and/or 
maintenance control supervisor, or coordination (or its lack) with other AMTs 
(sometimes including a Lead Mechanic) working on the same task. Of the eleven 
original cases, four of those cases were used “as is” for ASAP scenarios. Six of the 
eleven cases were provided a second ending, in addition to the “as is” conclusion of each 
case. One case was given two additional endings. The eleven original cases, plus the 
eight alternate endings produced a total of 19 individual potential ASAP scenarios. The 
additional endings usually took the form of the AMT reporter 1) questioning a 
supervisor’s request that he/she sign off the repair as airworthy, but then signing; or 2) 
having “second thoughts” after acquiescing to the directive, or 3) refusing to sign as 
airworthy. Two other alternate endings had the AMT reporter offering a reason why 
he/she thought the mistake in the case was made. 

The reason why most alternative endings were used was to examine the effects of 
signing under protest, refusing to sign, and signing with later misgivings. It was reasoned 
that ASAP programs should engender trust and AMT confidence in open communication 
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within maintenance. If that was true, then ASAP reports of 1) having “second thoughts 
about the wisdom of signing for a job, 2) protesting a supervisor’s instruction to sign for 
an unsafe act, and 3) challenging supervisor’s instruction by refusing to sign, might be 
forthcoming. The survey thus examined whether such scenarios would be accepted under 
ASAP. 

Before each scenario there was a question, “Was there an intentional disregard for 
safety?” Respondents were asked this question repeatedly in order to frame their answers 
in terms of the “community standard” (if there was one) for a risk-based philosophy. 
Respondents were not instructed further in how to answer, but they were assumed to be 
familiar with and capable of applying rule-based criteria (e.g., delay in reporting, sole 
source, etc.) as well, if they wished to do so. 

Each of the 19 scenarios required “respondent”, in his/her hypothetical role of a 
member of an ASAP Event Review Committee (ERC), to choose one of four actions: 

• Accept that report into the ASAP program as a self-disclosure of a possible 
violation of 14CFR, for detennining further action; 

• Accept that submission as a specific or general safety concern, for possible use 
identifying a system safety hazard. [This action choice is based on language in 
AC120-66 (Item 10, “ERC Process;” cf, FAA, 2002, Appendix 1, p.5) that states 
that among the reports an ERC might expect, would be those of a “general safety 
concern” that may or may not include a violation of regulations.] 

• Table it pending more information; 

• Reject or exclude it, — not allow the submission to be considered under the 
program. 
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For each scenario there was also space where the R could provide written comments 
about the case, or about his/her answer to it. 

The survey is included here as Appendix A. 

Administration . During October 2003 the survey form was submitted to the 
NASA ARC Fluman Research Institutional Review Board for approval. On November 
18 the board authorized the survey for use in research activities. Thereafter, the survey 
was posted on the Internet [cf., http://mrm.engr.scu.edu/actionsurvey.html ] for rapid and 
efficient data collection and analysis. The survey was anonymous, although respondents 
were asked to indicate whether they were answering from the point of view of an ERC 
member who represented management, the regulator, or an employee group or trade 
union. They were not asked any other personal information which might identify them 
later. 

Sample. The sample needed to be comprised of individuals with ample 
knowledge of maintenance ASAP rules as specified in FAA Advisory Circular 120-66, 
and who have the perspective of an ASAP user (as distinct from an ASAP policymaker). 
Thus the population was all individuals who have direct experience with administering a 
maintenance ASAP program of their own, or who are currently developing such a 
program. Since there were few maintenance ASAP programs (either in place or in 
development) in the U.S. during 2003 this population is small - probably numbering no 
more than 35 to 40 people. The potential respondents for the present study were 18 
individuals of my direct acquaintance, plus six others who were introduced to me during 
the study. These were people with profound ASAP knowledge and represented up to 
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two-thirds of that population at that time. They came from airline maintenance 
management, AMTs’ trade unions, and FAA CMOs. 

During October 2003 I visited or telephoned all 18 of the first set of individuals in 
the potential sample respondents and explained my study. I asked each one if they would 
be willing to provide me with an email address that I could use to send them the URL for 
the survey. They all agreed to review the scenarios and respond to the survey if they felt 
comfortable doing so. During December the 18 potential respondents from five airline 
maintenance organizations were formally invited to complete the web-based survey. 
Seven responses were received during December. As a follow-up, the same invitation 
was made again to the same 18 in January. Seven more completed surveys were 
received, for a total of fourteen during December 2003 and January 2004. This 
represented a response rate of 77%. 

In April 2004 I telephoned the manager of a sixth airline’s maintenance ASAP 
program. I described the study and asked him to endorse the survey to his ERC and their 
alternates. During May five respondents submitted their surveys, raising the overall 
sample size to 19 and the return rate to 78%. 

The total sample consists of six maintenance management representatives, five 
FAA inspectors, and eight trade union representatives. 


10 



Quantitative Results 


Dominant responses. The following section describes the results obtained from 
all 19 multiple-choice questions. Table 1 displays the overall summary results in terms 
of percent responses to each of the four multiple choice answers. 

Table 1 


Summary Percentages, All Scenarios, Dec-Jan Sample (n=19) 


Scenario # 

% Accept as 
Event 

% Accept as 

Hazard 

% Reject 

% Delay for 
additional 
information 

1 

42 

21 

11 

26 

2a 

37 

26 

5 

32 

2b 

53 

16 

5 

26 

2c 

47 

16 

5 

32 

3a 

90 

- 

10 

- 

3b 

74 

5 

5 

16 

4 

44 

28 

- 

28 

5a 

61 

- 

22 

21 

5b 

50 

11 

17 

22 

6a 

94 

- 

6 

- 

6b 

94 

- 

6 

- 

7 

56 

28 

6 

11 

8 

61 

- 

17 

22 

9a 

33 

22 

17 

28 

9b 

39 

28 

22 

11 

10a 

39 

22 

17 

22 

10b 

28 

28 

16 

28 

11a 

50 

22 

6 

22 

lib 

39 

33 

5 

22 


It will be seen from Table 1 that percentages in the first response column, “Accept 
as Event,” are typically higher than the other choices. That column, together with 
percentages in the second column, “Accept as Hazard,” sum to no less than 56% for each 
one of the 19 scenarios. Note also that percent “Reject” reaches as much as 22% for only 
two scenarios. These results provide prima facie evidence for a community standard of 
accepting errors caused by (or at least accompanied by) social pressure. 
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Another way of presenting these dominant responses is to sum the raw data 
contributing to the results for each of the columns in Table 1 and to calculate the overall 
percentages for each of the four response choices. Those results are as follows. Over all 
scenarios and respondents, “Accept the report as a safety event” was the most frequent 
(n=187, 54% of total) response. “Table the report for more information” was a distant 
second (n=68, 20% of total). Third in order was “Include it as a hazard report” (n=56, 
16% of total), and “Reject this report” was last (n=35, 10% of total). These findings 
suggests that for most of these respondents, most of the time, these 19 scenarios are 
largely clear - either “Reject” (10% of the time), or “Accept” (70% of the time, 
combining individual Event Reports and Hazard infonnation). For 20% of the total 
responses received, the information was incomplete or ambiguous enough to warrant 
further investigation (“Table it”). 

Intergroup differences across all scenarios. A Chi-Square test was applied to the 
proportion of each choice-point selected by each of the three respondent groups 
(Managers, Regulators, and Employee Representatives) represented in the present 
sample. Only two statistically significant differences (p. <.05) were found among the 19 
scenarios (#3 a, 1 la). Closer examination shows that among the three groups, the 
Regulator Representatives are slightly more likely to reject scenario 3a. On the other 
hand, scenario 1 la is called a “hazard report” by a disproportionate number of Employee 
Representatives. Open ended comments to scenario 11a reveal that “event report” should 
probably have been used instead. These two results may be statistical anomalies rather 
than important distinctions among the three groups of ERC representatives. Thus the 
overall pattern suggests that Managers, Regulators, and Employee Representatives, as 
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groups, do not respond differently to the scenarios (or to most of the scenarios) presented 
to them. 

Differences Among the 19 Scenarios 

Those scenarios for which there was the most agreement were those involving 
mistakes made while working with other AMTs. Those with the greatest disagreement 
were scenarios which involved some degree of supervisory pressure. 

Scenarios with highest agreement. Over a quarter of the scenarios (5/19) show 
extremely high agreement for acceptance. Frequency counts item by item across the 19 
scenarios show two (#6a-b) have almost complete unanimity (18/19 respondents accept 
the safety report), one (#3 a) has 17 accept, and two more (#3b & 7) have acceptance by 
15 respondents (either as Individual Reports or Hazard information). These five 
scenarios all deal with sign-off errors made by relying on a work mate to determine the 
correct part or procedure under time pressure. These five scenarios may be characterized 
as “honest mistakes, owned up to by the reporters.” 

The remaining 14 scenarios also received reasonably high acceptance, but 
somewhat lower than the five previously described. 

Scenarios #1 la, & lib had high acceptance (n=13, or 72%, of respondents 
accepted them either as “Event Reports” or “Hazard Reports”) and 11a and b each 
received one outright rejection plus four requests for more information. Those cases 
involved an AMT who, after performing inspection or repair, felt uneasy about the 
aircraft flying without written confirmation from engineering. Scenario #1 received 12 
acceptances, two outright rejections plus five requests for more infonnation. This case 
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was about an inspector who was concerned that his/her name was on the airworthiness of 
an aircraft that he/she wanted to undergo further checks. These three scenarios may be 
characterized as “management or system errors.” 


One scenario (#4) involved an AMT signing off repairs, which by inference also 
included results of prior poor practice — and because of which, failed to pass inspection. 
This case received high acceptance by 13 respondents (72%). Comments suggested that 
the reported had not violated 14 CFR. Five respondents chose “table it” and their 
comments indicated they would want to invite management to explain the prior repairs. 

Another scenario (#8) was concerned with an AMT who had received an FAA 
Letter of Investigation (LOI) after signing off an aircraft which had an air turnback 
because of the repair. This case received high acceptance (12 respondents or 61%), but 
four respondents chose “table it” and three respondents rejected it outright. Written 
comments supporting the rejections mentioned that having waited for the LOI the 
reporter was outside the time limits for submission (a rule-based decision). 

Scenario cases 9b and 10b both involved situations where an AMT refused to sign 
for airworthiness. One of them 9b received high acceptance (n=13, 63%), but also 
received four rejections. Examination of the comments for 9b showed that two of the 
rejections were aimed at the supervisor who signed, not at the AMT who reported it. 
Scenario 10b received 10 acceptances (56%), three rejections and four requests to table it. 
In this case there were very few comments written and none of them discussed the 
rejections. 
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The remaining seven scenarios (2a-c, 5a-b, 9a, 10a) all dealt with the issue of 
supervisory pressure and AMTs acquiescing to it. This “pressure” to sign for work 
resulted in several different outcomes — the submitter either claimed to have protested at 
the time he/she signed (2b, 5a, 9a, 10a); had signed and later had misgivings (2a, 5b) or 
had refused to sign and another AMT had done so (2c). It is important to note that even 
these seven scenarios received a clear majority of acceptance under the ASAP rules as 
interpreted by the present sample of informed and knowledgeable maintenance 
professionals. Two of these seven cases (2a, 2c) received acceptance from 63% of the 
respondents and were rejected by only one respondent each (5%). The others had slightly 
lower rates ranging from 61% (5a, 5b, 10a) to 56% (9a). The reject rates also varied from 
5% to 22% and the comments ranged from “Intentional disregard [for safety]” to, “The 
technician knowingly did what he was asked to do. Definitely lack of assertiveness,” and 
“The mechanic knew what he was doing was not correct but still did it.” This distinction 
between nearly two-thirds accept to nearly one-quarter reject for these cases provides the 
variability to promote good discussion during ASAP/ERC training while at the same time 
promoting support for the community standard of what is not intentional disregard for 
safety. 


Qualitative Results 

In total, survey respondents wrote comments 34 percent of the time. That is, in 
1 18 of the 346 possible opportunities to do so (19 respondents x 19 scenarios, less 15 
times questions were not answered) respondents wrote comments. In nearly every case, 
these comments explain or expand the multiple-choice answer respondent gave. In an 
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overwhelming number (some 95%) the comments confirm the multiple-choice response 
and show a comprehensive understanding of the ASAP intention and process. 

Comments indicating “false negatives.” In those few (6) remaining cases the 
comments reveal ambiguity in the case scenario itself, or just random error. These 
comments describe the intention to accept a scenario (either as a “safety hazard report” or 
“event report”) or “table it,” despite check-marking the “reject” answer. Most of these 
illuminating comments were in response to scenarios dealing with supervisors placing 
pressure on AMTs to agree to sign for work the latter believe to be incomplete or 
incorrect. These comments said something like, I reject the AMT’s submission, because 
the supervisor [“. . .is at fault”], or should be. . . [“. . .interviewed to get their side of the 
story”], or [“. . .asked to submit their own ASAP event about this case”]. This clear 
discrepancy between what an respondent check-marked and what he/she said in 
associated comments can be considered a misunderstanding, or mistake. These few 
“mistakes” in check-marking an answer were all in the direction of “false negatives” - so, 
if anything, the large acceptance of the 19 scenarios revealed by the quantitative results is 
slightly strengthened by the “accepting” comments written in by the respondents written 
in association with selecting the “reject” choice. 

Implications for Training 

The results on acceptance of these 19 scenarios confirm that any and all of them 
reflect acceptability under community standards for protected disclosure under aviation 
maintenance ASAP. Thus any of the scenarios could be used to heighten the appreciation 
of present and potential ERC members to the complicating effects of social context in 
risk-based decisions regarding ASAP event submissions. 
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The Three ASAP Roles. Furthermore the results suggest that there are few 
differences in the acceptance threshold of regulator, management and employee ERC 
representatives in the present sample. The three groups respond to these scenarios in 
much the same manner and can be considered together for the purposes of community 
standards. Whether these three groups might be more effectively trained, using these 
scenarios, within their own groups, or with the other groups cannot be answered with the 
data here. 

Which scenarios to use? What is of interest for the present study is exploring the 
possibility of ERC confusion caused by the example of an excluded case in AC 120-66B, 
in which a supervisor directs a mechanic to use an available lubricant , which is known to 
them both to be incorrect (FAA, 2002; p 13). For present training purposes then, an 
efficient approach would be to limit the number of scenarios to those most similar to that 
case in AC 120-66B, and which display enough response variability in the present sample 
to insure spirited discussion among ASAP trainees. Seven scenarios (# 2a, 2b, 2c, 5a, 5b, 
9a, and 10a) fit those criteria. 

Modifying the scenarios for use in training. The training method chosen here 
takes the form of interactive discussion familiar in adult learning. Specifically the 
training using these scenarios would take the fonn of individual pre-work, followed by 
small group discussion, then summarized (either by trainees or trainers) for lessons 
learned and affirmation by the participants. Appendix B contains the seven scenarios, as 
training cases, plus written instructions for training. 
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Conclusion 


ASAP scenarios have been developed from publicly available aviation 
maintenance safety cases. These scenarios have been tested with respondents in Air 
Carriers and FAA CMOs who belong to Maintenance ASAP ERCs or are currently 
planning to implement a Maintenance ASAP program. The scenarios have proven 
realistic and appropriate to the respondents. There is agreement, across management, 
FAA, and trade union respondents, that real cases like these scenarios should be included 
as ASAP self-disclosure events (or as safety hazards) to provide organizational learning 
and improvement in risk management. 

Most of the scenarios developed for this study are suitable for use in Maintenance 
ASAP training. A subset of seven cases has been chosen as particularly relevant to the 
issue of supervisory pressure on AMTs. These scenarios have been edited and expanded 
to act as stand-alone training materials for use in educating ERC members. 

There are issues that lie outside this report. For one, the results reported above do 
not prove that these maintenance ERC members can recognize and agree when they see 
non-inadvertent disregard for safety. What is reported above is not a full examination of 
acceptance-rejection criteria for maintenance ASAP. Only one-half of the picture (albeit 
an important one) has been discovered here. Perhaps the future will bring further 
research that specifies the community standards for ASAP rejection as well as 
acceptance. 

The results above do show is that we are beginning to understand the “community 
standards” in aviation maintenance that deal with “unintentional disregard for safety.” 
There is little doubt that these respondents certainly knew it when they didn't see 
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intentional disregard for safety - or they were willing to risk giving the reporter the 
benefit of the doubt. 
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Appendix A: The Survey 


ASAP SURVEY PARTICIPATION 

We are studying the type of incidents that are accepted in Maintenance Aviation 
Safety Action Partnership (ASAP) programs. This study is being conducted under the 
NASA Aviation Program. Your professional experience is invaluable in helping us 
understand the line between acceptable and unacceptable cases and how the best 
maintenance information can be collected through ASAP. We value your participation. 

There are no risks in participating in this survey. However, you should know that 
your participation is voluntary and that if you decide not to participate in this study, or 
any part of the study, at any time, it will not affect your relationship with the airline that 
employs you, your union, or NASA. All information provided by you will remain 
anonymous and confidential. We do not ask for nor should you include any personal 
information with your responses (this includes your name, your airline, your location, 
employee number, etc.). Final results of this study may be published but your individual 
responses will remain anonymous and confidential. Final results will also be available to 
you upon request. If you wish to communicate with me directly, my email is below. 

Thank you for your time and effort in helping us with this study. It is genuinely 
appreciated! Your knowledge and experience has the potential to benefit the industry by 
influencing future training and operating guidelines. 

Sincerely, 

James C. Taylor, 

QSS 

NASA ARC 

Moffett Field, CA 94035 
Phone# (650) 604-5056 
Email, j taylor@mail. arc .nasa. gov 
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ASAP SURVEY: INSTRUCTIONS 


Following, you will find a set of 1 1 case scenarios written in the form of ASAP report. 
You will see that some of the case scenarios have at least one variation. I ask you to 
consider each version of each case for its applicability to an ASAP program that your 
organization would sponsor. To answer these questions typically requires only 10-15 
minutes of your time and your feedback will provide us with unique and meaningful data 
for recommending training and operations standards for the industry. 
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ASAP SURVEY: BACKGROUND INFORMATION 


Before getting started evaluating the case scenarios, please tell us which part of the 
ASAP partnership you would represent by selecting the appropriate choice below 

Commercial Airline employing mechanics 

Repair Station employing mechanics 

Trade Union or Employee Group representing mechanics 

Regulator 
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ASAP SURVEY: CASE STUDIES 
Case Study #1 


1. Intentional disregard for safety? I was working on the C-check for a B737-300. 
Yesterday I was assigned a routine inspection of the horizontal stabilizer jackscrew 
and gimbal. Although the wear was well within limits (.020”) I wrote up a non- 
routine because of metal particles I noted on the jackscrew. After I went home my 
supervisor called to ask me if the metal debris I wrote up was sheet metal from a 
previous modification or drill bit shavings. I said it could be, but I was not sure - 
this is why I made the write-up. Supervisor then used my name as a ref. to sign off 
the non-routine card saying I said it was sheet metal debris. I did not give him 
permission to use my name or say I said it was sheet metal debris. He had not 
gotten my permission to use my name as a reference. They also did not do further 
inspection on gimbal/jackscrew for damage or perform an ops check for 
airworthiness. 

If you were a member of an ASAP Event Review Committee (ERC), what would 
you vote to do with report #1? 

Please choose only ONE: 

a) Accept it into your ASAP program as a safety event report 

b) Accept it into your ASAP program it as a safety hazard report 

c) Table it while more information is collected 

d) Reject it as unacceptable for ASAP 

2. Intentional disregard for safety? I am an inspector with six years experience on 
third shift heavy maintenance at XXX. I was working on an MD80, assigned a 
routine inspection of the elevator jackscrew nut. It measured just within wear 
limits (.035” < .040”). I wrote up the discrepancy and called for replacing the part. 
The next day my supervisor asked me to take additional measurements (because the 
first one was so close) and my measurements still remained within wear limits (.034” 
-.037”). And... 

[Ending A:] ...at my supervisor’s instruction I signed the task card as 
airworthy, but I am now concerned that the wear tolerance won’t allow continued 
safe flight until next regular inspection in two years. 

If you were a member of an ASAP Event Review Committee (ERC), what would you vote 
to do with report #2 A? 

CHOOSE ONE: 

Accept it into your ASAP program as a safety event report 

Accept it into your ASAP program it as a safety hazard report 

Table it while more infonnation is collected 

Reject it as unacceptable for ASAP 
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[Ending B:] ...at first I didn’t want to change my discrepancy calling for 
replacing the part, and I tried to explain why, but at my supervisor’s instruction I 
signed the task card as airworthy. 

If you were a member of an ASAP Event Review Committee (ERC), what would you 
vote to do with report #2B? 

CHOOSE ONE: 

Accept it into your ASAP program as a safety event report 

Accept it into your ASAP program it as a safety hazard report 

Table it while more infonnation is collected 

Reject it as unacceptable for ASAP 

[Ending C:] ...I refused to sign the task card as airworthy, but my supervisor 
asked another mechanic to measure it again and sign it off. I am still concerned that 
the wear tolerance won’t allow continued safe flight until next regular inspection in 
two years. 

If you were a member of an ASAP Event Review Committee (ERC), what would you 
vote to do with report #2C? 

CHOOSE ONE: 

Accept it into your ASAP program as a safety event report 

Accept it into your ASAP program it as a safety hazard report 

Table it while more infonnation is collected 

Reject it as unacceptable for ASAP 
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3. Intentional disregard for safety? My lead and I were called to a through flight 
for a #2 tank xfeed problem on a 727-200. We noticed that the switch on the FE’s 
panel would not open or close the valve - it was stripped out. As we called to get 
dispatch approval per MEL, I proceeded to remove the cannon plug on the x-feed 
valve. Remembering the switch on the FE’s panel being in closed position I moved 
the valve manually to the opposite side and lock wired it, but before I did this my 
lead ran up to the cockpit to verify that it was the valve to be deferred per MEL. 
My lead was taking care of the paperwork while I was finishing up. The plane was 
dispatched, but 30 minutes out they turned back not being able to fuel #1 and #3 
engines with #2 tank. The problem being the valve was lock wired closed. And... 

[Ending A:] ...I feel that this problem was caused because of the rush and 
pressure of a through flight. 

If you were a member of an ASAP Event Review Committee (ERC), what would you 
vote to do with report #3 A? 

CHOOSE ONE: 

Accept it into your ASAP program as a safety event report 

b) Accept it into your ASAP program it as a safety hazard report 

cl Table it while more information is collected 

Reject it as unacceptable for ASAP 

[Ending B:] ...I feel that this problem was caused by not reading the whole 
procedure of the MEL. A very important part was missed - to transfer fuel out of 
the #2 tank to verify that the valve was in the open position. 

If you were a member of an ASAP Event Review Committee (ERC), what would you 
vote to do with report #3B? 

CHOOSE ONE: 

Accept it into your ASAP program as a safety event report 

Accept it into your ASAP program it as a safety hazard report 

Table it while more infonnation is collected 

Reject it as unacceptable for ASAP 
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4. Intentional disregard for safety? I am an A&P mechanic working in the 
fiberglass/composites shop. I have 11 years with the company and 18 months in the 
shop. I repair DC9 radomes, tail cones, ADF and VOR antenna. I have informed 
Shop management, the company’s training personnel and QA about the use of 
unapproved methods in our shop, and have been directed by them NOT to remove 
repairs done in the past, by someone else, that I felt had been done improperly. 
These repairs are leading to failures in Radomes when they are tested for radar 
transmission efficiency. Radome serial number xxxx is one such, it was recently 
repaired by me and it failed in the test shop in places other than the place I 
repaired. I heard that management has released this radome to service because the 
location of the flaws in the radar presentation to the pilot. My repairs did not 
contribute to that failure. It failed the test because of unapproved methods done 
before by others in the shop. Management has even brought in the manufacturer’s 
engineer to approve the present method and revise the SRM. This request was 
denied by the manufacturer, as I understand it, because the present unapproved 
repair allows no damage tolerance and would be a problem with a rear engined 
aircraft. 

If you were a member of an ASAP Event Review Committee (ERC), what would you vote 
to do with report #4? 

CHOOSE ONE: 

Accept it into your ASAP program as a safety event report 

Accept it into your ASAP program it as a safety hazard report 

Table it while more infonnation is collected 

Reject it as unacceptable for ASAP 
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5. Intentional disregard for safety? The LMP of this B757-200 had been 
downgraded to “Not CAT II” due to right autopilot and thrust management on 
MEL. On day 10 of the MEL I fixed the autopilot yet the thrust management 
system remained unrepaired. I was instructed by management to clear the placard 
and reissue it using the authority to restart the clock. And... 

[Ending A:] ...I did it, but under protest. I told them that this is in violation 
of GPM XX-xx, page x. 

If you were a member of an ASAP Event Review Committee (ERC), what would you 
vote to do with report #5 A? 

CHOOSE ONE: 

Accept it into your ASAP program as a safety event report 

Accept it into your ASAP program it as a safety hazard report 

Table it while more infonnation is collected 

Reject it as unacceptable for ASAP 

[Ending B:] ...although I did it, I was uncomfortable that maybe I was non- 
compliant with the letter of the MEL. 

If you were a member of an ASAP Event Review Committee (ERC), what would you vote 
to do with report #5B? 

CHOOSE ONE: 

Accept it into your ASAP program as a safety event report 

Accept it into your ASAP program it as a safety hazard report 

Table it while more infonnation is collected 

Reject it as unacceptable for ASAP 
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6. Intentional disregard for safety? Clearing MEL for “No Heat Light” on TAT 
(DADC) probe for a 737-400. I installed wrong part # that was given to me by my 
lead (he read it from the old part removed). Aircraft was dispatched, but air 
turnback with “B” Autopilot problem, which I deferred. After a/c departed 2 nd time 
I rechecked effectivity screen and realized the probe I replaced was the wrong one. 

I notified MtcControl and correct part was installed on next leg and “B” autopilot 
squawk was cleared. And... 

[Ending A:] ...I thought the p/n was checked and in the rush of clearing the 
MEL on a through flight a lack of communication caused an error. 

If you were a member of an ASAP Event Review Committee (ERC), what would you vote 
to do with report #6 A? 

CHOOSE ONE: 

Accept it into your ASAP program as a safety event report 

Accept it into your ASAP program it as a safety hazard report 

Table it while more infonnation is collected 

Reject it as unacceptable for ASAP 

[Ending B:] ...my lead later told me that the p/n he obtained from the old 
part was almost unreadable and in the rush of clearing the MEL on a through flight 
a lack of communication caused an error. 

If you were a member of an ASAP Event Review Committee (ERC), what would you vote 
to do with report #6B? 

CHOOSE ONE: 

Accept it into your ASAP program as a safety event report 

Accept it into your ASAP program it as a safety hazard report 

Table it while more infonnation is collected 

Reject it as unacceptable for ASAP 

7. Intentional disregard for safety? Day shift through flight. Yesterday, myself and 
another mechanic changed nose wheel tires on a B737 while our Lead checked parts 
tags for both tires and completed log book entry. We were in a hurry to get the a/c 
out and both of us mechanics failed to notice that the tires were different and the 
lead failed to notice that although the parts numbers were the same for the wheels, 
the tires had different parts effectivity for different versions of the 737. The 
aircraft was dispatched with one incorrect tire, which the captain noticed before 
departing on the next leg. The foreman told us about the error today. 

If you were a member of an ASAP Event Review Committee (ERC), what would you vote 
to do with report #7? 

CHOOSE ONE: 

Accept it into your ASAP program as a safety event report 

Accept it into your ASAP program it as a safety hazard report 

Table it while more infonnation is collected 

Reject it as unacceptable for ASAP 
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8. Intentional disregard for safety? I am a lead mechanic on the line at XXX. We 
received an air return of a 737 with #2 Engine Heat Light on. Light came on for less 
than a minute. Another AMT and I looked for loose or leaking ducts and 
fouled/rubbing overheat sensors, but found nothing. Subsequent power run up OK, 
and no 30 day history of “air return,” so I cleared logbook and dispatched the 
aircraft. Problem recurred after takeoff. Later, on third shift, another AMT found 
9 th stage duct leak because clamp was missing its bolts, gasket, and safety wire. I’ve 
been sent a LOI because I signed the logbook to clear the aircraft. 

If you were a member of an ASAP Event Review Committee (ERC), what would you vote 
to do with report #8? 

CHOOSE ONE: 

Accept it into your ASAP program as a safety event report 

Accept it into your ASAP program it as a safety hazard report 

Table it while more infonnation is collected 

Reject it as unacceptable for ASAP 

9. Intentional disregard for safety? Today an Airbus A320 arrived at our station 
with a totally deflated nose landing gear strut. History showed identical condition 
at previous station where “quick service” was performed. Maintenance manual 
requires full service at next maintenance opportunity. Aircraft was scheduled for 
light service at our station. Plight was delayed for strut service. Myself and another 
mechanic believed full service was required, but station maintenance manager 
insisted that we only perform a “quick check.” Strut was serviced with nitrogen 
and released and dispatched. And... 

[Ending A:] ...although the other mechanic and I protested to our 
supervisor, I did sign off the logbook 

If you were a member of an ASAP Event Review Committee (ERC), what would you vote 
to do with report #9 A? 

CHOOSE ONE: 

Accept it into your ASAP program as a safety event report 

Accept it into your ASAP program it as a safety hazard report 

Table it while more infonnation is collected 

Reject it as unacceptable for ASAP 

[Ending B:] ...the other mechanic and I refused to sign the logbook, so the 
supervisor signed it off 

If you were a member of an ASAP Event Review Committee (ERC), what would you vote 
to do with report #9B? 

CHOOSE ONE: 

Accept it into your ASAP program as a safety event report 

Accept it into your ASAP program it as a safety hazard report 

Table it while more infonnation is collected 

Reject it as unacceptable for ASAP 


29 



10. Intentional disregard for safety? A DC9-40 was being converted to all cargo 
configuration and I was the inspector assigned to this a/c on day shift. Maintenance 
was working an engineering authorization (EA). At about 1415 hours it was 
decided to stop work on this job due to the shift ending. I suggested that 
maintenance complete a job pass-down form as required by the GMM when the 
work procedure is interrupted. The maintenance supervisor objected and wanted to 
sign the steps we had completed. I explained that I could not stamp steps 5a 
through e as those steps configured the a/c for the test, and was not repeated 
anywhere on the EA. We then went to the second shift inspection supervisor who 
agreed that a job pass-down should be used. The maintenance supervisor then said 
he would hold the EA and have none of it inspected rather than attach a job pass- 
down to it. At that point the inspection supervisor did a turn-about and heavily 
pressured me to stamp off the work steps for the a/c configuration. And... 

[Ending A:] ...after protesting about it I did stamp the a/c configuration and 
a job pass-down was not used. 

If you were a member of an ASAP Event Review Committee (ERC), what would you vote 
to do with report #10 A? 

CHOOSE ONE: 

Accept it into your ASAP program as a safety event report 

Accept it into your ASAP program it as a safety hazard report 

Table it while more infonnation is collected 

Reject it as unacceptable for ASAP 

[Ending B:] ...I refused to stamp off the a/c configuration and the inspection 
supervisor did it. 

If you were a member of an ASAP Event Review Committee (ERC), what would you 
vote to do with report #10B? 

CHOOSE ONE: 

Accept it into your ASAP program as a safety event report 

Accept it into your ASAP program it as a safety hazard report 

Table it while more infonnation is collected 

Reject it as unacceptable for ASAP 
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11. Intentional disregard for safety? On July XX a DC-10 30 XYZ inbound with 
discrepancy, “DIRECTIONAL CTL PANEL HDG DISPLAY DIGITS OUT OL 
SEQUENCE.” After ordering the correct part number, part was not available. I 
was told by maintenance control to rob the panel from a/c YMZ. That panel had a 
different p/n and showed not interchangeable with a/c XYZ. Me, and the supervisor 
on duty, discussed the situation with maintenance control. Maintenance control in 
turn decided to get engineering involved and after a while my supervisor was told 
that the panel from a/c YM Z was applicable for a/c XYZ. I installed the part and 
performed the ops check and all checks were good. And... 

[Ending A:] ...I signed it off and the aircraft was returned to service. I have 
asked my supervisor to see the engineering approval for the part interchangeability, 
but he hasn’t seen it either. It’s been 2 weeks now and a/c XYZ has flown without 
fit guidance panel problems, but it may be out there with an unapproved part. 

If you were a member of an ASAP Event Review Committee (ERC), what would you vote 
to do with report #11 A? 

CHOOSE ONE: 

Accept it into your ASAP program as a safety event report 

Accept it into your ASAP program it as a safety hazard report 

Table it while more infonnation is collected 

Reject it as unacceptable for ASAP 

[Ending B:] ...I refused to sign it off, so the supervisor did. It’s been 2 weeks 
now and a/c XYZ has flown without fit guidance panel problems, but I’m 
conncerned that it may be out there with an unapproved part. 

If you were a member of an ASAP Event Review Committee (ERC), what would you vote 
to do with report #11B? 

CHOOSE ONE: 

Accept it into your ASAP program as a safety event report 

Accept it into your ASAP program it as a safety hazard report 

Table it while more infonnation is collected 

Reject it as unacceptable for ASAP 
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Appendix B: Maintenance ASAP Risk-Based Decision Training Materials 


Part I. Introducing the Concepts of “Risk-Based” Decisions, and “Community 
Standards” in Maintenance ASAP. 20 Minutes 

The trainer begins a 20 minute lecture module with a summary of “rule-based” 
and “risk-based” criteria for ASAP decisions. 

This is followed by an introduction to “community standards” for ideas important 
to a community (such as aircraft maintenance), but which are difficult to define. Prime 
example in ASAP is “intentional disregard for safety.” Even though such an idea may be 
difficult to formally define, community members will “know it when we see it.” 

Finally, the Trainer summarizes (the present) NASA research into community 
standards for risk-based decisions in aviation maintenance ASAP. 
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Part II. Individual pre-work - “on your own.” 20 Minutes 


Here are seven examples of possible cases you may see in your role of member in 
your Maintenance ASAP Event Review Committee (ERC). Read each case and mark 
your answer to it before going on to the next one. After you have read and responded to 
all the cases you will have a chance to discuss the cases with other members of today’s 
class. 

A The LMP of this B757-200 had been downgraded to “Not CAT II” due to right 
autopilot and thrust management on MEL. On day 10 of the MEL I fixed the 
autopilot yet the thrust management system remained unrepaired. I was instructed 
by management to clear the placard and reissue it using the authority to restart the 
clock. And... 


[Ending for Case A- 1:] ...I did it, but under protest. I told them that this is 
in violation of GPM XX-xx, page x. 

As a member of the ASAP Event Review Committee (ERC), what would you vote to do 
with report Al? 

CHOOSE ONE: 

Accept it into your ASAP program as a safety event report 

Accept it into your ASAP program it as a safety hazard report 

Table it while more information is collected 

Reject it as unacceptable for ASAP 

[Ending for Case A-2:] ...although I did it, I was uncomfortable that maybe I 
was non-compliant with the letter of the MEL. 

As a member of the ASAP Event Review Committee (ERC), what would you vote to do 
with report #A2? 

CHOOSE ONE: 

Accept it into your ASAP program as a safety event report 

Accept it into your ASAP program it as a safety hazard report 

Table it while more information is collected 

Reject it as unacceptable for ASAP 
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Case B. Today an Airbus A320 arrived at our station with a totally deflated nose 
landing gear strut. History showed identical condition at previous station where 
“quick service” was performed. Maintenance manual requires full service at next 
maintenance opportunity. Aircraft was scheduled for light service at our station. 
Flight was delayed for strut service. Myself and another mechanic believed full 
service was required, but station maintenance manager insisted that we only 
perform a “quick check.” Strut was serviced with nitrogen and released and 
dispatched. And although the other mechanic and I protested to our supervisor, I 
did sign off the logbook 

As a member of the ASAP Event Review Committee (ERC), what would you vote to do 
with report B? 

CHOOSE ONE: 

Accept it into your ASAP program as a safety event report 

Accept it into your ASAP program it as a safety hazard report 

Table it while more information is collected 

Reject it as unacceptable for ASAP 


Case C. A DC9-40 was being converted to all cargo configuration and I was the 
inspector assigned to this a/c on day shift. Maintenance was working an engineering 
authorization (EA). At about 1415 hours it was decided to stop work on this job 
due to the shift ending. I suggested that maintenance complete a job pass-down 
form as required by the GMM when the work procedure is interrupted. The 
maintenance supervisor objected and wanted to sign the steps we had completed. I 
explained that I could not stamp steps 5a through e as those steps configured the a/c 
for the test, and was not repeated anywhere on the EA. We then went to the second 
shift inspection supervisor who agreed that a job pass-down should be used. The 
maintenance supervisor then said he would hold the EA and have none of it 
inspected rather than attach a job pass-down to it. At that point the inspection 
supervisor did a turn-about and heavily pressured me to stamp off the work steps 
for the a/c configuration. And after protesting about it I did stamp the a/c 
configuration and a job pass-down was not used. 

As a member of the ASAP Event Review Committee (ERC), what would you vote to do 
with report C? 

CHOOSE ONE: 

Accept it into your ASAP program as a safety event report 

Accept it into your ASAP program it as a safety hazard report 

Table it while more information is collected 

Reject it as unacceptable for ASAP 
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D. I am an inspector with six years experience on third shift heavy maintenance at 
XXX. I was working on an MD80, assigned a routine inspection of the elevator 
jackscrew nut. It measured just within wear limits (.035” < .040”). I wrote up the 
discrepancy and called for replacing the part. The next day my supervisor asked 
me to take additional measurements (because the first one was so close) and my 
measurements still remained within wear limits (.034” - .037”) . And... 

[Ending Case D-l:] ...at my supervisor’s instruction I signed the task card as 
airworthy, but I am now concerned that the wear tolerance won’t allow continued 
safe flight until next regular inspection in two years. 

As a member of the ASAP Event Review Committee ( ERC ), what would you vote to do 
with report D1 ? 

CHOOSE ONE: 

Accept it into your ASAP program as a safety event report 

Accept it into your ASAP program it as a safety hazard report 

Table it while more information is collected 

Reject it as unacceptable for ASAP 

[Ending Case D-2:] ...at first I didn’t want to change my discrepancy calling 
for replacing the part, and I tried to explain why, but at my supervisor’s instruction 
I signed the task card as airworthy. 

As a member of the ASAP Event Review Committee (ERC), what would you vote to do 
with report D2? 

CHOOSE ONE: 

Accept it into your ASAP program as a safety event report 

Accept it into your ASAP program it as a safety hazard report 

Table it while more information is collected 

Reject it as unacceptable for ASAP 

[Ending Case D-2:] ...I refused to sign the task card as airworthy, but my 
supervisor asked another mechanic to measure it again and sign it off. I am still 
concerned that the wear tolerance won’t allow continued safe flight until next 
regular inspection in two years. 

As a member of the ASAP Event Review Committee (ERC), what would you vote to do 
with report D3? 

CHOOSE ONE: 

Accept it into your ASAP program as a safety event report 

Accept it into your ASAP program it as a safety hazard report 

Table it while more information is collected 

Reject it as unacceptable for ASAP 
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Part III. Small group discussion. 25 Minutes 


You will now join the other members at your table to discuss one of these seven 
ASAP cases. Your instructor will tell you which case your group will work on first. 

Your instructor will also appoint three of you as ERC members (the others are “alternate” 
members and will not participate in the discussion itself). The ASAP Manager in each 
group will chair the discussion and report to the class what the decision was. The result 
of this group discussion will be a joint decision on the action that the ERC will take. 

That decision will be reached by consensus (no “majority rules” votes, no “horse 
trading”). Your group will have 15 minutes for discussion and consensus. The ASAP 
manager will have five minutes to announce and explain the decision reached to the class 
as a whole. 

Before the group begins you will choose support roles for the “alternate” 
members. First of all, choose a “scribe” - someone who will keep notes on the flip chart 
provided. Next choose a “timekeeper.” Finally choose an “observer” who will comment 
on the group’s process. After the ASAP Manager has explained the decision reached, the 
observer will have 5 minutes to report back to the class on how the discussion progressed 
as well as what when well and what went poorly. 

Part IV. The Large Group Discussion. 25 Minutes 

After all the small groups have presented their decisions and their process, the 
Trainer opens the floor to questions and comments. The Trainer lists those comments on 
flip chart. The Trainer then summarizes the issues and concerns that are reflected in the 
larger groups. 
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The Trainer asks if there are other concerns/issues that have not yet been raised. 
Those are listed separately on flip chart. 

Finally the Trainer combines the decisions presented from the small groups, with 
the large group’s issues, and relates these to the results of the NASA research. The 
Trainer gains consensus from the larger group about accepting ASAP cases dealing with 
possible supervisory pressures for improving organizational learning and proactive safety 
management. 
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